1. Field
The present disclosure relates to communications in computer networks. More particularly, this invention is directed toward a virtualization of a co-processor data plane.
2. Description of Related Technology
In computer systems, virtualization is a process by which a virtual version of computing resources, such as hardware and software resources, i.e., a central processor unit, a storage system, an input/output resources, a network resource, an operating system, and other resources known in the art, are simulated by a computer system, referred to as a host machine.
FIG. 1 depicts a conceptual structure of a virtualization system 100. A hardware platform 102, comprises all physical entities embodying computing resources required by a specific host machine, i.e., a central processor unit, an input/output resources, a storage system, a network resource, and other resources known to a person having ordinary skill in the art. To avoid undue complexity, only a storage system 104, a network resource 106, a System Memory Management Unit (SMMU) 108, and a Memory Management Unit (MMU) 110 are shown. The storage system 104, may comprise a hard drive, a semiconductor based memory, and other types of memory known in the art. The terms storage system and memory are used interchangeably. The network resource 106 may comprise at least one Network Interface Card (NIC) or other network interface entity known to a person of ordinary skills in the art.
The hardware platform 102, together with an optional software entity 112, i.e., operating system, comprises a host machine operating a Type 2 hypervisor, also known as hosted hypervisor 114. As well known to a person having ordinary skill in the art, the optional software entity 112 is not necessary for Type 1 hypervisor, also known as native hypervisor. A hypervisor is software or firmware entity that creates and operates at least one virtual machine, also referred to as a guest and/or a guest machine. As depicted in FIG. 1, the hosted hypervisor 114 created and operates three virtual machines 116. Through hardware virtualization, the hosted hypervisor 114 provides each virtual machine 116 with a virtual hardware operating platform. By interfacing with the virtual hardware operating platform, the virtual machines 116 access the computing resources of the host machine to execute the virtual machines' respective operations. As a result, a single host machine can support multiple virtual machines 116, each operating an operating system (not shown) and/or other software entity, i.e., an application (not shown), simultaneously through virtualization.
To enable transfer of data into and from the virtualization system 100, via network resource 106, as well as among different entities of the virtualization system 100, the hosted hypervisor 114, the virtual machines 116, and the optional software entity 112, instantiate a data plane entity 118. A data plane entity comprises a firmware or a software entity executed on the hardware platform 102 underlying the guests, i.e., the hosted hypervisor 114 and the virtual machines 116. Additionally, the data plane may be executed by a user process within a virtual machine 116, on which the user process executes.
In a typical host machine, the virtual hardware operating platform should be presented to the virtual machines in a manner that assures that the virtual nature of the hardware platform should not be discernible to the virtual machines. Consequently, the host machine should avoid conflicts between virtual machines in accessing the computing resources. To accomplish these goals, the host machine may implement a translation scheme between the virtual machines' software and the host machine's resources. One of such translation schemes is accomplished via uses of the SMMU 108 and/or the MMU 110, as explained in greater detail in reference to FIG. 2.
Consider FIG. 2, depicting an example of a conceptual architecture of a virtualized system 200. The architecture comprises a plurality of co-processors, assisting the processor cores 228 with functions comprising, e.g., packet input 220, packet output 222, memory allocation 224, work scheduling, 226, and other functions known to a person of ordinary skills in the art. A co-processor comprises a processing unit used to supplement the functions of the central processing unit. Supplemental functions, performed by the co-processor, may comprise floating point arithmetic, graphics, signal processing, string processing, encryption, input/output interfacing with peripheral devices, and other functions known to a person of ordinary skills in the art. The co-processor carries out these functions under a close control of a supervisory processing unit, e.g., the central processing unit. A processor core is a processing unit at the central processing unit, which reads and executes program instructions.
Data in form of a packet 230 arrives at the network interface 206, and is parsed by a packet input 220. The packet input 220 determines a guest-pool identifier 232, in accordance with one of the fields comprising the parsed packed, e.g., an inbound MAC address, a quality-of-service information, and/or other fields known to a person or ordinary skills in the art. Although the packet input 220, comprising a hardware, or hardware and software entity, is depicted as a part of the network interface 206, in another aspect the packet input 220 may be implemented as separate entity. Guest-pool is a virtual resource assigned to a guest. A virtual pool comprises a number of blocks in a virtual memory assigned to the guest.
Each guest-pool is associated with a quality of service that a packet stored in the guest-pool should receive. Quality of service is the overall performance of the network communication, including the transfer of data within the virtualized system, as seen by the users, and is quantified by measuring different parameters, e.g., error rates, bandwidth, throughput, transmission delay, availability, jitter, and other parameters known to persons of ordinary skills in the art.
The packet input 220 then requests a memory allocator 224 to allocate a space in the memory 204 to which to write the parsed packet. In an aspect implementing a quality of service, the request comprises the guest-pool identifier 232, which indicate a guest-pool (not shown) for the parsed packet, so that a required quality of service for the parsed packet may be achieved. The memory allocator 224 translates the guest-pool identifier 232 into a corresponding local-pool identifier (not shown), indicating the local-pool 234 in the memory 204, and allocates the requested space in the local-pool 234. A local-pool comprises a number of blocks, i.e., portion of the (physical) memory 204. The packet input 220 writes the parsed packet to the requested space in the local-pool 234 by translating a virtual address of a space in the guest-pool via a System Memory Management Unit (SMMU) 208.
The SMMU 208 comprises a hardware or a hardware and software entity interacting with the co-processors (220, 222, 224, 226) and the memory 204, to guarantee that the memory 204 is properly virtualized, i.e., that a specific virtual address can only access resources in the memory 204, to which the owner of the virtual address has been granted access. Such a guarantee may be accomplished by utilizing, e.g., a stream identifier, and/or transaction identifier, and/or context identifier, or any other mechanisms known to a person of ordinary skill in the art. The term stream indicates a specific flow of information between a virtual entity requesting a memory transaction, i.e., writing to or reading from the physical memory, and determines the address space in the physical memory to which the corresponding virtual memory transaction belongs.
The SMMU 208 then uses the identifier together with the virtual address in the guest-pool provided by the packet input 220 to compute a physical address in a corresponding local-pool 234 in the memory 204 and writes the parsed packet to the local-pool 234.
The packet input 220 then informs a scheduler 226 that work 236 is needed to be carried out. Work is a general concept of something to be processed. Work properties are specific to an implementation of a scheduler. By means of an example, such work properties may comprise a work queue entry, comprising a pointer to the work to be processed. By means of an example, in the case of an interrupt work, the pointer indicates a data structure used to process the interrupt. Another property may comprise a work quality of service, i.e., identifier(s) which portion of the scheduler will handle the work, processor cores that may handle the work, priority related to that work, and other quality of service related identifier(s) known to person of ordinary skills in the art. Yet another property may indicate whether the work can be carried out in parallel manner, or whether the work must be carried out in serial manner. A person of ordinary skills in the art will understand that other and/or additional work properties are contemplated.
The scheduler 226 schedules the work 236 for a processor core, e.g., a processor core 228_1 at a virtual machine 216_1 that is assigned to carry out the work 236 on the parsed packet. In one aspect, the scheduler 226 may need to maintain data structures organizing work in the memory 204, and therefore allocate or return pointers to the memory 204 from the memory allocator 224 and/or access the memory 204.
The processor core 228_1 requests the parsed packet via a MMU 210. The MMU 210 comprises a hardware, or a hardware and software entity that guarantees that a software executing on any of the processor cores 228 can only access resources of any of the co-processors (220, 222, 224, 226) and the memory 204 to which the processor core 228 was granted access, by a supervising entity, e.g., the hypervisor 214.
The mechanism used by the MMU 210 to supervise access to resources for the processor cores 228, may comprise the mechanism used by the SMMU 208 to supervise access to resources for the co-processors (220, 222, 224, 226).
However, not to hinder performance, certain communicative connections, e.g., among the co-processors: packet input 220, packet output 222, and scheduler 226, as well as between the co-processors: packet input 220, packet output 222, scheduler 226, and the memory allocator 224, in the architecture of a virtualized system 200 are not protected by either the MMU 210 nor the SMMU 208. Consequently, a rouge or errant guest software could gain access to functions, e.g., request work from a scheduler 226, or access region in a memory 204 via a memory allocator 224, to which that guest software is not authorized. Although the guest software, which runs on processor core(s) 228, is communicating via MMU 210 via communicative coupling 229, the MMU 210 safeguards only an address. Thus, when the guest software issues an address of a co-processor, e.g., the scheduler 226, with which the guest software is authorized to communicate, the MMU 210 allows the communication, and the guest software may then instruct the scheduler 226 to access memory allocator 224, to which the guest software is not authorized. Consequently, current architectures are not able to avoid all conflicts between virtual machines in accessing the computing resources.
Additionally, the unprotected communicative connections mean that guests do not have a private numbering space when communicating virtual resource assigned to the guest. Ideally each guest should be under the illusion that the virtual resource assigned to the guest is unique. By means of an example the guest should be under the illusion that a guest-pool with identifier 0, assigned to the guest is unique and isolated from a guest-pool with identifier 0, assigned to another guest, even though in the hardware memory 204, there is only a single local-pool with identifier 0 and not a local-pool with identifier 0 for each guest.
Accordingly, there is a need in the art for a co-processor data plane virtualization, providing a solution to at least the above identified problems.